<html>
<head><meta charset="utf-8"><title>Test data format · t-cargo/PubGrub · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/index.html">t-cargo/PubGrub</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html">Test data format</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="212478176"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212478176" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212478176">(Oct 06 2020 at 20:05)</a>:</h4>
<p>We need to choose a format to keep our test data in. The data is mostly going to consist of 1) a list of all packages with their published versions 2) dependency trees for those packages; the format should accommodate those. It should be human readable, so binary formats can probably be safely skipped altogether.</p>
<p>Some choices that come to mind:<br>
1) TOML as it is used in Cargo. Not sure if it's a proper choice due to the flat structure.<br>
2) JSON. Common choice with performant parsers available. Downsides of having no comments (major) and trailing commas (minor) ask to look into other options.<br>
3) RON. Solves the JSON downsides listed above. As it was created specifically with Rust interoperability in mind, it should provide natural mapping to the actual structs in the code.</p>
<p>Feel free to comment on the options from the list and bring up new suggestions. If someone actually used RON and has some experience to share, I'd be glad to hear.</p>



<a name="212482956"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212482956" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthieu Pizenberg <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212482956">(Oct 06 2020 at 20:41)</a>:</h4>
<p>Never use RON myself. Seems a good choice, but <span class="user-mention" data-user-id="120179">@Eh2406</span> has probably the best insight on this.</p>



<a name="212486888"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212486888" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eh2406 <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212486888">(Oct 06 2020 at 21:15)</a>:</h4>
<p>I thought I would like toml. but the way it stored a crate with 2 dependents was a no go. Comments are needed, so I am working from RON for now. Thanks for the sugestion!</p>



<a name="212488712"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212488712" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212488712">(Oct 06 2020 at 21:32)</a>:</h4>
<p>You are welcome.</p>
<p>If someone else sees this thread and has something to add I'd be glad to hear from anyone. :)</p>



<a name="212493815"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212493815" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eh2406 <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212493815">(Oct 06 2020 at 22:26)</a>:</h4>
<p>Here is what I have so far. For example <code>fn no_conflict()</code>'s solver is encoded as:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="s">"root"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="s">"1.0.0"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="s">"foo"</span>: <span class="p">(</span><span class="w"></span>
<span class="w">                </span><span class="n">segments</span>: <span class="p">[</span><span class="w"></span>
<span class="w">                    </span><span class="p">(</span><span class="s">"1.0.0"</span><span class="p">,</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="s">"2.0.0"</span><span class="p">)),</span><span class="w"></span>
<span class="w">                </span><span class="p">],</span><span class="w"></span>
<span class="w">            </span><span class="p">),</span><span class="w"></span>
<span class="w">        </span><span class="p">},</span><span class="w"></span>
<span class="w">    </span><span class="p">},</span><span class="w"></span>
<span class="w">    </span><span class="s">"foo"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="s">"1.0.0"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="s">"bar"</span>: <span class="p">(</span><span class="w"></span>
<span class="w">                </span><span class="n">segments</span>: <span class="p">[</span><span class="w"></span>
<span class="w">                    </span><span class="p">(</span><span class="s">"1.0.0"</span><span class="p">,</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="s">"2.0.0"</span><span class="p">)),</span><span class="w"></span>
<span class="w">                </span><span class="p">],</span><span class="w"></span>
<span class="w">            </span><span class="p">),</span><span class="w"></span>
<span class="w">        </span><span class="p">},</span><span class="w"></span>
<span class="w">    </span><span class="p">},</span><span class="w"></span>
<span class="w">    </span><span class="s">"bar"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="s">"2.0.0"</span>: <span class="p">{},</span><span class="w"></span>
<span class="w">        </span><span class="s">"1.0.0"</span>: <span class="p">{},</span><span class="w"></span>
<span class="w">    </span><span class="p">},</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>



<a name="212518911"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212518911" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212518911">(Oct 07 2020 at 06:19)</a>:</h4>
<p>Looks pretty clean</p>



<a name="212540668"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212540668" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthieu Pizenberg <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212540668">(Oct 07 2020 at 10:43)</a>:</h4>
<p>The <code>(segments: [...])</code> is a rather literal mirror of the <code>Range</code> implementation. Since dependency constraints are more restricted (not the result of intersections and unions of ranges) would it make more sense to have a special codec that generate things more similar to what is usable in <code>Cargo.toml</code>, or maybe more systematic since every correct dependency can be expressed in this form maybe <code>1.0.0 &lt;= v &lt; 1.4.5</code></p>



<a name="212540977"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212540977" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212540977">(Oct 07 2020 at 10:46)</a>:</h4>
<p>I agree it would be great to have it more general to allow us to change the representation in the future.</p>



<a name="212541647"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212541647" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212541647">(Oct 07 2020 at 10:54)</a>:</h4>
<p>Regarding <code>1.0.0 &lt;= v &lt; 1.4.5</code>, I'm a bit cautious if we are about to reinvent our own semver that uses explicit ranges only? If so, does it make sense to do it or are we better off using the semver spec itself?</p>



<a name="212556818"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212556818" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eh2406 <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212556818">(Oct 07 2020 at 13:21)</a>:</h4>
<p>some minor improvements:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="p">(</span><span class="w"></span>
<span class="w">    </span><span class="n">dependencies</span>: <span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="s">"bar"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="s">"1.0.0"</span>: <span class="p">{},</span><span class="w"></span>
<span class="w">            </span><span class="s">"2.0.0"</span>: <span class="p">{},</span><span class="w"></span>
<span class="w">        </span><span class="p">},</span><span class="w"></span>
<span class="w">        </span><span class="s">"foo"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="s">"1.0.0"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">                </span><span class="s">"bar"</span>: <span class="p">[</span><span class="w"></span>
<span class="w">                    </span><span class="p">(</span><span class="s">"1.0.0"</span><span class="p">,</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="s">"2.0.0"</span><span class="p">)),</span><span class="w"></span>
<span class="w">                </span><span class="p">],</span><span class="w"></span>
<span class="w">            </span><span class="p">},</span><span class="w"></span>
<span class="w">        </span><span class="p">},</span><span class="w"></span>
<span class="w">        </span><span class="s">"root"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="s">"1.0.0"</span>: <span class="p">{</span><span class="w"></span>
<span class="w">                </span><span class="s">"foo"</span>: <span class="p">[</span><span class="w"></span>
<span class="w">                    </span><span class="p">(</span><span class="s">"1.0.0"</span><span class="p">,</span><span class="w"> </span><span class="nb">Some</span><span class="p">(</span><span class="s">"2.0.0"</span><span class="p">)),</span><span class="w"></span>
<span class="w">                </span><span class="p">],</span><span class="w"></span>
<span class="w">            </span><span class="p">},</span><span class="w"></span>
<span class="w">        </span><span class="p">},</span><span class="w"></span>
<span class="w">    </span><span class="p">},</span><span class="w"></span>
<span class="p">)</span><span class="w"></span>
</code></pre></div>



<a name="212558703"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212558703" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eh2406 <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212558703">(Oct 07 2020 at 13:34)</a>:</h4>
<p>And I am feeling "perfect is the enemy of good" vibes. I'd love to improve the range representation, get rid of the <code>[</code> if there is only one item, but I don't want to add a lot of potentially buggy parsing and deparsing  code while still working on adding other needed testing. <br>
Then again writing it to a file was easier than I expected, so it may be worth a quick try.</p>



<a name="212559841"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212559841" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212559841">(Oct 07 2020 at 13:43)</a>:</h4>
<p>99% of the time it's going to be 1 item, might be worth hiding the array if it's not too much of a hustle.<br>
Cargo doesn't even support disjoint ranges, does it?</p>



<a name="212559891"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212559891" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212559891">(Oct 07 2020 at 13:43)</a>:</h4>
<p>Let me know if you need some help with this btw</p>



<a name="212560639"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212560639" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eh2406 <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212560639">(Oct 07 2020 at 13:48)</a>:</h4>
<p>I know there is talk of adding and "or" syntax to semver. Do you know what is it? As you sead, we may as well be valid semver.</p>



<a name="212564174"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212564174" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthieu Pizenberg <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212564174">(Oct 07 2020 at 14:14)</a>:</h4>
<p>My personal impression (that you may have deduced already ^^) is that I don't like much all the ranges syntax of semver that we see in the wild. I don't think the semver spec itself specifies anything more than what versions means. To my understanding, those ranges syntaxes are mostly there to make it easier to write configuration files in the npm ecosystem which popularized them. But the exact meaning of caret <code>^</code> or tilde <code>~</code> for example is not clear at all until you read somewhere what they mean. What I like about things like <code>v1 &lt;= v &lt; v2</code> is that it works whatever the system of versions you are using, numbers, semantic versions, strings, whatever as long as they have an order. That's why I proposed this as an example. It's not as much as reinventing semver, but rather staying as close as possible to the meaning of our version trait. At least that's how I see it, but that's not me doing the work ^^</p>



<a name="212564793"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212564793" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthieu Pizenberg <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212564793">(Oct 07 2020 at 14:18)</a>:</h4>
<p><span class="user-mention silent" data-user-id="120179">Eh2406</span> <a href="#narrow/stream/260232-t-cargo.2FPubGrub/topic/Test.20data.20format/near/212558703">said</a>:</p>
<blockquote>
<p>And I am feeling "perfect is the enemy of good" vibes. I'd love to improve the range representation, get rid of the <code>[</code> if there is only one item, but I don't want to add a lot of potentially buggy parsing and deparsing  code while still working on adding other needed testing. </p>
</blockquote>
<p>That makes perfect sense!</p>



<a name="212565745"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212565745" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212565745">(Oct 07 2020 at 14:25)</a>:</h4>
<p>I like math notation for specifying ranges, e.g. "[1, 2)" for 1 inclusive, 2 exclusive. It gets rid of some arbitrary symbol in the middle ("v") and is quite established.<br>
Not sure how it fits in RON file though, where [] are for arrays and () are for tuples. Might make it confusing.<br>
What do you think guys?</p>



<a name="212577284"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212577284" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eh2406 <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212577284">(Oct 07 2020 at 15:38)</a>:</h4>
<blockquote>
<p>Then again writing it to a file was easier than I expected, so it may be worth a quick try.</p>
</blockquote>
<p>Poked at it this morning, and did not get a solution I was happy with. So I am leaning to making these improvements, in some other PR.</p>



<a name="212619743"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212619743" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthieu Pizenberg <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212619743">(Oct 07 2020 at 21:10)</a>:</h4>
<p><span class="user-mention silent" data-user-id="340766">Alex Tokarev</span> <a href="#narrow/stream/260232-t-cargo.2FPubGrub/topic/Test.20data.20format/near/212565745">said</a>:</p>
<blockquote>
<p>I like math notation for specifying ranges, e.g. "[1, 2)" for 1 inclusive, 2 exclusive. It gets rid of some arbitrary symbol in the middle ("v") and is quite established.</p>
</blockquote>
<p>Did you meant "[1, 2["? the ")" is rather for directions so more or less infinite here. Like it to :) and the <code>None</code> case, which corresponds to infinite could be something like "[1,[" with an empty second slot.</p>
<blockquote>
<p>Poked at it this morning, and did not get a solution I was happy with</p>
</blockquote>
<p>Makes sense to improve this later then</p>



<a name="212627106"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212627106" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212627106">(Oct 07 2020 at 22:29)</a>:</h4>
<p>Where I studied we used round braces to specify "exclusive". First time I see the reverse square bracket.<br>
Might just be country specific.</p>



<a name="212627223"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212627223" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212627223">(Oct 07 2020 at 22:31)</a>:</h4>
<p>[1, ∞)<br>
That's how infinity would be indicated</p>



<a name="212968820"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/212968820" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eh2406 <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#212968820">(Oct 11 2020 at 15:08)</a>:</h4>
<p>btw, just found <a href="https://github.com/afilip1/ronfmt">https://github.com/afilip1/ronfmt</a> it works great for benchmark files!</p>



<a name="213075822"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/260232-t-cargo/PubGrub/topic/Test%20data%20format/near/213075822" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Tokarev <a href="https://rust-lang.github.io/zulip_archive/stream/260232-t-cargo/PubGrub/topic/Test.20data.20format.html#213075822">(Oct 12 2020 at 19:10)</a>:</h4>
<p>Nice find! I was looking for Intellij plugin to do that, found one that was released a few versions ago and I couldn't use that. I thought of maybe trying to upgrade it, but your solution solves the problem right away <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>